The present invention relates to the testing of logic designs in simulation or hardware, and more particularly to a method and system for measuring and reporting the extent of test coverage of a design-under-test (DUT) in terms of its bus transactions.
A process known as verification comprises applying test cases to a DUT and checking the results against expected results to determine whether the DUT functions correctly. Functional verification of logic designs which interface to standardized buses typically involves the use of a software tool known as a bus functional model (BFM). A BFM emulates a bus protocol to apply a test case in simulation to a DUT modeled by a hardware description language (HDL).
The BFMs allow verification engineers to write test cases as bus transactions. The bus transactions may be expressed in bus functional language (BFL) statements which are decoded and executed by the BFMs using a logic simulator. For example, a standardized BFM transaction test case for a 32 bit data slave design that uses read and write transactions with address, data, and control signal information could comprise BFL statements appearing as follows (comments indicated by xe2x80x9c//xe2x80x9d):
In the foregoing example, read and write are two different transaction types, while address, byte enable, and data are parameters of the respective transaction types. The parameters were used to specify data to be written to an address, and to subsequently read the address to determine whether the data was correctly written.
In order to verify that a DUT completely complies with a standardized bus interface, all possible bus transactions must be performed to ensure that the DUT-to-bus interface operates under all legal bus conditions. This process, also known as regression testing, entails the enumeration of all possible and legal transaction combinations to ensure a complete and correct design.
The testing of a DUT for bus compliance can be a complex and time-consuming procedure, requiring considerable computer resources. Numerous bus transactions and varied test scenarios must typically be applied. Often, executing a complete set of tests that exhausts all the possible bus transactions of a DUT at one time is impracticable. More typically, test cases must be applied in limited batches. Accordingly, tracking of the extent of test coverage for a design over a testing period to ensure that complete test coverage has been achieved becomes a difficult chore that slows the regression testing or verification process.
In view of the foregoing, a method and system for conveniently measuring and reporting the extent of test coverage is needed.
In a method and system according to the invention, incremental test coverage information from an application of a test case to a DUT is collected. The incremental test coverage information, along with cumulative test coverage information, is automatically correlated with a set of valid bus transactions corresponding to the DUT to determine the extent of the test coverage for the DUT.
In an embodiment, the correlation process uses a DUT configuration file which describes the DUT in a condensed syntax. The correlation process automatically generates possible bus transactions for the DUT from the syntax. Rules included in the syntax filter out selected ones of the possible bus transactions to produce the set of valid transactions. The valid transactions are compared with the incremental and cumulative test coverage information to determine which valid bus transactions have or have not been applied in testing the DUT.
In view of the foregoing, the present invention provides a convenient and easy-to-use measure of test coverage. The invention allows a user to apply relatively small sets of tests and incrementally build toward total coverage. A user can combine the information of multiple test case applications, either in simulation or hardware, and measure not only what percentage of coverage has been achieved, but what remaining transactions are needed to accomplish full coverage.